Quality control apparatus, moving image transmission system, quality control method, and recording medium

ABSTRACT

A moving image transmission system determines adequate number of packets, which are corresponding to redundant portion, in accordance with a network state. 
     The moving image transmission system estimates a correlation between a transmission bit rate and a packet loss rate based on both of information which indicates the packet loss rate and bit rate information, and determines the packet loss rate, which makes an encode bit rate of moving image data maximum, based on the correlation, and determines redundancy, which is applied to the moving image data, based on the determined packet loss.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a National Stage of International Application No.PCT/JP2010/070642 filed Nov. 15, 2010, claiming priority based onJapanese Patent Application No. 2009-266018 filed Nov. 24, 2009, thecontents of all of which are incorporated herein by reference in theirentirety.

TECHNICAL FIELD

The present invention relates to a quality control method for a movingimage.

BACKGROUND ART

In the case that a moving image transmission apparatus uses a networkwhose network bandwidth is not secured sufficiently for providing theinternet service and the mobile communication service, an availablenetwork bandwidth of the moving image transmission apparatus becomesunstable due to various causes. There is a moving image transmissionapparatus to transmit contents, such as a moving image, which requirehigh level throughput continuously, through the network. The movingimage transmission apparatus suffers from frequent stop of playing backthe contents due to shortage of the network bandwidth. Hereinafter, itis defined that the moving image includes any one or both of a movingpicture and a voice.

As one of general methods to solve the above-mentioned problem in thisfield, there is a method that the moving image transmission apparatuscarries out data compensation for a packet loss by use of Forward ErrorCorrection (FEC) which has a fixed redundancy. In the case thatcharacteristics of the network, which is used for transmitting themoving image, is known in advance, it is possible to compensate the datain real time with a simple method of using FEC. However, in the casethat many packet losses exceeding a packet loss rate which is estimated,in advance, on the basis of the network characteristics are caused, themoving image transmission apparatus can not compensate the lost packet.Accordingly, the moving image transmission apparatus has a defect thatit is difficult to use FEC in the network whose available bandwidth ischanged significantly. In order to avoid the problem, the moving imagetransmission apparatus assigns firstly large redundancy to data relatedto a service, and then provides the service. In this case, the movingimage transmission apparatus transmits the packet excessively even in astate that the network has almost no problem. Accordingly, the movingimage transmission apparatus consumes the network bandwidth more thannecessity.

As another general method to solve the above-mentioned problem in thisfield, there is a method that a moving image receiving apparatusrequests to re-send lost data and the moving image transmissionapparatus re-sends the requested data to improve immunity against thepacket loss. In the case that the method is used, the moving imagetransmission apparatus has an advantage that the network bandwidth,which is required for re-sending the data, is narrow, since the movingimage transmission apparatus re-sends only the lost data. However, inthe case that the method is used, it takes the following time forre-sending the data, that is, a time until the moving image transmissionapparatus recognizes that the packet loss is caused, a time requisite torequesting to resend the data, and a time requisite to re-sending thelost data. Accordingly, in the case that the method is used, longlatency is caused in an interval from transmitting the data firstlyuntil regenerating the data. As a result, a defect that the real-timecharacteristic is lost is caused.

As a method, which remedies the defect of FEC, out of theabove-mentioned methods, a method denoted as Adaptive FEC is proposed ina non-patent literature (NPL) 1. The moving image transmissionapparatus, which uses the Adaptive FEC, changes the redundancydynamically so as to minimize the packet loss. Accordingly, it ispossible that the moving image transmission apparatus restrains thepacket loss even if the moving image transmission apparatus communicatesthrough the network whose available bandwidth is changed to some extent.Moreover, in the case that the network is in a good state, it ispossible that the moving image transmission apparatus transfers themoving image with making the redundancy not so large. As a result, it ispossible to reduce an ineffective use of the network bandwidth.

However, according to the method described in the non-patent literature1, the moving image transmission apparatus makes the redundancy large inthe case that the network enters into a bad state. For this reason, themoving image transmission apparatus transmits and receives excessivelythe packets which include an amount of data corresponding to theredundancy. Therefore, according to the method described in thenon-patent literature 1, there is a possibility that the moving imagetransmission apparatus falls into a bad cycle of making the networkstate degraded furthermore. As a method to improve the problem, there isa method which is described in a patent literature 1. According to thepatent literature (PTL) 1, the moving image transmission apparatusreduces packets which carry data of the moving image, and determines anencoding bit rate of the moving image so that a bandwidth, which iscorresponding to a total number of packets including the data of themoving image and packets including redundancy data of FEC, may benarrower than the network bandwidth. As a result, it is possible thatthe moving image transmission apparatus obtains the best quality of themoving image without degrading the network quality.

According to further another method described in a patent literature(PTL) 2, in a situation that the network state becomes degraded, themoving image transmission apparatus carries out transmitting andreceiving the packet with giving priority to the packet carrying themoving image, and restrains to transmit the packet based on FEC. As aresult, even if the network state becomes degraded abruptly, the movingimage transmission apparatus can prevent that the network state becomesdegraded furthermore due to the packet which the moving imagetransmission apparatus transmits and receives.

PRECEDING TECHNICAL LITERATURE Patent Literature

-   -   [PTL 1]: Japanese Patent Application Laid-Open No. 2006-262288    -   [PTL 2]: Japanese Patent Application Laid-Open No. 2005-175837

Non Patent Literature

-   -   [NPL 1]: Shimizu, et al. “Implementation of Adaptive FEC by        dynamic re-configurable system”, Information Processing Society        of Japan DA symposium 2003, pp. 25 to 30. 2003/July

BRIEF SUMMARY OF THE INVENTION Problem to be Solved by the Invention

The related arts mentioned above have a problem that the moving imagetransmission apparatus can not determine adequate number of the packets,which are corresponding to a redundant portion, based on correlationbetween the encoding bit rate and the packet loss rate of the movingimage. The reasons are as follows. According to the art described in thenon-patent literature 1, number of the packets, which are correspondingto the redundant portion, is determined on the basis of the networkstate. Therefore, according to the art described in the non-patentliterature 1, there is a possibility that the packet, which iscorresponding to the redundant portion and which is transmitted andreceived by the moving image transmission apparatus, furthermore makesthe network congested frequently, and consequently the packet loss rateis degraded. Moreover, according to the art described in the patentdocument 1, it is assumed that the moving image transmission apparatususes all the available network bandwidth, and a change of the packetloss rate due to a change of the encode bit rate of the moving image isnot taken into the consideration. Accordingly, in the case that thereare a plurality of apparatuses which compete with each other for thenetwork and a plurality of the apparatuses transmit and receive aplurality of the moving images simultaneously, it is difficult that themoving image transmission apparatus estimates an adequate networkbandwidth which the moving image transmission apparatus can use.According to the technology described in the patent literature 2, tochange the encoding bit rate of the moving image is not taken into theconsideration. Therefore, according to the art described in the patentliterature 2, the packet loss rate may be lowered in some cases throughnot transmitting and not receiving the packet which is added through themoving image transmission apparatus applying FEC, but there is apossibility that the image quality is degraded furthermore due to thepacket loss.

One of the objects of the present invention is to provide a qualitycontrol apparatus or the like which determines adequate number ofpackets, which are corresponding to the redundant portion, in accordancewith a correlation between the encoding bit rate and the packet lossrate of the moving image.

Means to Solve the Problem

A first quality control apparatus according to an exemplary embodimentof the present invention includes:

a state information receiving unit to receive network state informationwhich includes both of a packet loss rate of moving image data caused ata certain time and time information indicating the certain time;

a state information record storing apparatus to store bit rateinformation, which includes both of a transmission bit rate of themoving image data caused at a certain time and time informationindicating the certain time, in association with the network stateinformation which includes the time information; and

a moving image quality control unit to read both of informationindicating the packet loss rate which is associated with the timeinformation included in each packet of the moving image data and thetransmission bit rate from the state information record storingapparatus, and to estimate a correlation between an encode bit rate andthe packet loss rate of the moving image data based on each of the readinformation, and to determine the packet loss rate, which makes theencode bit rate of the moving image data maximum, based on thecorrelation, and to determine redundancy, which is applied to the movingimage data, based on the determined packet loss rate.

A first moving image transmission system according to an exemplaryembodiment of the present invention includes a moving image transmissionapparatus and a moving image receiving apparatus.

In the above configuration, the moving image transmission apparatusincludes:

a transmission unit which transmits moving image data at a predeterminedtransmitting bit rate;

a state information receiving unit to receive network state informationwhich includes both of a packet loss rate of the moving image datacaused at a certain time and time information indicating the time;

a state information record storing apparatus to store bit rateinformation, which includes both of the transmission bit rate of themoving image data used at a certain time and time information indicatingthe time, in association with the network state information whichincludes the time information; and

a moving image quality control unit to read both of the informationindicating the packet loss rate which is associated with the timeinformation included in each packet of the moving image data and thetransmission bit rate from the state information record storingapparatus, and to estimate a correlation between an encode bit rate andthe packet loss rate of the moving image data based on each of the readinformation, and to determine the packet loss rate, which makes theencode bit rate of the moving image data maximum, based on thecorrelation, and to determine redundancy, which is applied to the movingimage data, based on the determined packet loss rate.

The moving image receiving apparatus includes:

a data receiving unit which receives the moving image data from themoving image transmission apparatus; and

a state information transmitting unit to transmit both of the packetloss rate of the moving image data and time information indicating atime, at which the moving image transmission apparatus transmits eachthe packet included in the moving image data, to the moving imagetransmission apparatus.

A first quality control method according to an exemplary embodiment ofthe present invention includes:

receiving network state information which includes both of a packet lossrate of moving image data caused at a certain time and time informationindicating the time; making a storage apparatus store bit rateinformation, which includes both of a transmitting bit rate of themoving image data used at a certain time and time information indicatingthe time, in association with the network state information whichincludes the time information;

reading both of the information indicating the packet loss rate which isassociated with the time information included in each packet of themoving image data and the transmitting bit rate from the storage medium;

estimating a correlation between an encode bit rate and the packet lossrate of the moving image data based on each of the read information;

determining the packet loss rate, which makes the encode bit rate of themoving image maximum, based on the correlation; and

determining redundancy, which is applied to the moving image data, basedon the determined packet loss rate.

A non-transitory recording medium according to an exemplary embodimentof the present invention stores a first program which makes a computerexecute:

a process to receive network state information which includes both of apacket loss rate of moving image data caused at a certain time, and timeinformation indicating the certain time;

a process to make a storage medium store bit rate information, whichincludes both of a transmitting bit rate of the moving image data usedat a certain time and time information indicating the certain time, inassociation with the network state information which includes the timeinformation;

a process to read both of the information indicating the packet lossrate, which is associated with the time information included in each thepacket of the moving image data and the transmission bit rate from thestorage medium;

a process to estimate a correlation between an encode bit rate and thepacket loss rate of the moving image data based on each of the readinformation;

a process to determine the packet loss rate, which makes the encode bitrate of the moving image data maximum, based on the correlation; and

a process to determine redundancy, which is applied to the moving imagedata, based on the determined packet loss rate.

Effect of the Invention

According to the moving image transmission apparatus of the presentinvention, it is possible to determine the adequate number of thepackets, which are corresponding to the redundant portion, in accordancewith the correlation between the encode bit rate and the packet lossrate of the moving image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a configuration according to a firstexemplary embodiment of the present invention.

FIG. 2 is a flowchart exemplifying an operation of a moving imagetransmission apparatus 101 according to the first exemplary embodimentof the present invention.

FIG. 3 is a flowchart exemplifying an operation of a moving imagereceiving apparatus 102 according to the first exemplary embodiment ofthe present invention.

FIG. 4 is a part of a flowchart exemplifying an operation of a movingimage quality control apparatus 105 according to the first exemplaryembodiment of the present invention.

FIG. 5 is a part of a flowchart exemplifying an operation of a movingimage quality control apparatus according to the first exemplaryembodiment of the present invention.

FIG. 6 shows an example of a relation between a transmission bit rateand a packet loss rate.

FIG. 7 shows an example of the relation between the transmission bitrate and the packet loss rate.

FIG. 8 shows an example of the relation between the transmission bitrate and the packet loss rate.

FIG. 9 shows an example of the relation between the transmission bitrate and the packet loss rate.

FIG. 10 shows an example of the relation between the transmission bitrate and the packet loss rate.

FIG. 11 is a block diagram showing a configuration according to a secondexemplary embodiment of the present invention.

FIG. 12 is a flowchart exemplifying an operation of a moving imagetransmission server 401 according to the second exemplary embodiment ofthe present invention.

FIG. 13 shows an example of a first correlation formula.

FIG. 14 shows an example of a second correlation formula.

FIG. 15 is a flowchart exemplifying an operation of a moving imagereceiving terminal 402 according to the second exemplary embodiment ofthe present invention.

FIG. 16 is a block diagram showing a modified example of the secondexemplary embodiment of the present invention.

FIG. 17 is a block diagram showing a configuration according to a thirdexemplary embodiment of the present invention.

EXEMPLARY EMBODIMENT TO CARRY OUT THE INVENTION

An exemplary embodiment according to the present invention will bedescribed in detail in the following. Here, components which have thesame function are assigned the same code in each exemplary embodimentwhich is described in each drawing and the specification.

First Exemplary Embodiment

FIG. 1 is a block diagram exemplifying a configuration of a moving imagetransmission system 1 according to a first exemplary embodiment of thepresent invention.

The moving image transmission system 1 includes a moving imagetransmission apparatus 101 and a moving image receiving apparatus 102.The moving image transmission apparatus 101 and the moving imagereceiving apparatus 102 are connected each other via a network 113 and anetwork 114.

Moving Image Transmission Apparatus 101

The moving image transmission apparatus 101 includes a state informationreceiving unit 103, a state information record storing apparatus 104, amoving image quality control unit 105, a moving image encoding unit 106,a FEC encoding unit 107 and a data transmitting unit 108. The movingimage encoding unit 106, the FEC encoding unit 107 and the datatransmitting unit 108 compose a transmission unit which transmits movingimage data at a predetermined transmission bit rate. A configuration ofthe transmission unit is not limited to the configuration shown inFIG. 1. The transmission unit may operate as a transmission circuitbased on control by CPU or operations as another hardware circuit. Thestate information receiving unit 103 is connected with the stateinformation record storing apparatus 104. The state information recordstoring apparatus 104 is connected with the state information receivingunit 103, the moving image quality control unit 105 and the datatransmitting unit 108. The moving image quality control unit 105 isconnected with the state information record storing apparatus 104, themoving image encoding unit 106 and the FEC encoding unit 107. The movingimage encoding unit 106 is connected with the moving image qualitycontrol unit 105 and the FEC encoding unit 107. The FEC encoding unit107 is connected with the moving image quality control unit 105, themoving image encoding unit 106 and the data transmitting unit 108. Thedata transmitting unit 108 is connected with the state informationrecord storing apparatus 104 and the FEC encoding unit 107.

State Information Receiving Unit 103

The state information receiving unit 103 receives network stateinformation on the network 113 via the network 114. The network stateinformation may include information which indicates a packet transmittedby the moving image transmission apparatus 101. Moreover, the networkstate information may include information which indicates a packet lossrate caused before FEC decoding, information which indicates the packetloss rate caused after the FEC decoding, or a cyclic counter value.

The information, which indicates the packet transmitted by the movingimage transmission apparatus 101, may be a time stamp value whichindicates a time when the packet is transmitted or the time stamp valuewhich indicates a relative time from a time to start transferring themoving image. Moreover, the information, which indicates the packettransmitted by the moving image transmission apparatus 101, may be asequence number assigned to each the packet. That is, it is enough thatthe information, which indicates the packet, can specify a time wheneach the packet is transmitted by the moving image transmissionapparatus 101. The information which can specify the time is also calledtime information.

The information which indicates the packet loss rate caused before theFEC decoding may be number of data which can not be received (lost data)by the moving image receiving apparatus 102 within a fixed period oftime. Moreover, the information which indicates the packet loss ratecaused before the FEC decoding may be a ratio of number of the packets,which the moving image receiving apparatus 102 can not receive normally,to number of all the packets which the moving image receiving apparatus102 receives. For example, the state information receiving unit 103 maycalculate the packet loss rate caused before the FEC decoding by use ofnumber of data which are transmitted by the moving image transmissionapparatus 101 within a fixed period of time, and number of data whichthe moving image receiving apparatus 102 loses within the fixed periodof time.

The information which indicates the packet loss rate caused after theFEC decoding may be number of data, which can not be decoded even by useof FEC, out of the data which the moving image receiving apparatus 102receives within a fixed period of time. Moreover, the information whichindicates the packet loss rate caused after the FEC decoding may be aratio of number of data, which the moving image receiving apparatus 102can not decode even by use of FEC, to number of all the packets whichthe moving image receiving apparatus 102 receives. For example, thestate information receiving unit 103 may calculate the packet loss ratecaused after the FEC decoding based on number of the data which themoving image receiving apparatus 102 receives, and number of the datawhich the moving image receiving apparatus 102 can not decode out of thereceived data even by use of FEC.

The cyclic counter value, which is assigned to each packet, is within apredetermined range. The counter value is assigned to each packetsuccessively in an order of the packet. In the case that the count valuereaches one end value of the predetermined range, a following packet isassigned the other end value of the predetermined range. It may beconfigured so that the data transmitting unit 108 may assign the cycliccounter value to each the packet.

State Information Record Storing Apparatus 104

The state information record storing apparatus 104 receives the networkstate information from the state information receiving unit 103.Moreover, the state information record storing apparatus 104 storesnetwork state record information. The network state record informationis generated by associating the network state information which thestate information record storing apparatus 104 receives, andtransmission bit rate information which is corresponding to theinformation indicating the packet included in the network stateinformation. The transmission bit rate information is generated byassociating the transmission bit rate of data which the moving imagetransmission apparatus 101 transmits, and a time at which the data istransmitted at the transmission bit rate.

The data transmitting unit 108, which will be described later, makes thestate record storing apparatus 104 store the transmission bit rateinformation.

Moving Image Quality Control Unit 105

Firstly, the moving image quality control unit 105 reads the networkstate record information on the packets, which the moving imagetransmission apparatus 101 transmits within a predetermined period oftime, out of the network state record information which the stateinformation record storing apparatus 104 stores. Specifically, themoving image quality control unit 105 determines whether a time, whichis determined by the information indicating the packet included in eachnetwork state record information, is within the predetermined period oftime. Moreover, the moving image quality control unit 105 specifies thatthe network state record information including information on the time,which is within the predetermined period of time, is the network staterecord information which the moving image quality control unit 105reads. In the case that the information which indicates the packet isthe time stamp value, the moving image quality control unit 105 maydetermine whether a time which the time stamp value indicates is withinthe predetermined period of time. Moreover, in the case that theinformation which indicates the packet is the sequence number, themoving image quality control unit 105 may determine with the followingmethod whether the information which indicates the packet is within thepredetermined period of time. First of all, the data transmitting unit108 generates information which associates the sequence number of eachthe packet, and the time at which the moving image transmissionapparatus 101 transmits the packet corresponding to the sequence number.Then, the moving image quality control unit 105 may determine on thebasis of the information generated by the data transmitting unit 108whether the information which indicates the packet is within thepredetermined period of time.

Secondly, the moving image quality control unit 105 estimates a firstcorrelation formula by determining a correlation between thetransmission bit rate and the packet loss rate, which are included ineach the network state record information, on the basis of the networkstate record information which is read. For example, the moving imagequality control unit 105 may determine the first correlation formulawith the minimum square error method.

In the case that denotation “correlation between transmission bit rateand packet loss rate” is used in the specification, the denotation“packet loss rate” means any one of the packet loss rate caused beforethe FEC decoding and the packet loss rate caused after the FEC decoding.

The moving image quality control unit 105 estimates an encode bit rateof the moving image, which the moving image transmission apparatus 101transmits, based on the transmission bit rate which is determined by thefirst correlation formula. For example, the moving image quality controlunit 105 estimates the encode bit rate of the moving image based on thedetermined transmission bit rate, a header size of the moving image dataat the transmission bit rate, and a data size of FEC data which isredundant data of FEC. Specifically, the moving image quality controlunit 105 may estimate the encode bit rate by subtracting a bit ratewhich is corresponding to data size of the header, and a bit rate whichis corresponding to data size of the FEC data from the transmission bitrate.

Thirdly, the moving image quality control unit 105 estimates a secondcorrelation formula by determining a correlation between the estimatedencode bit rate of the moving image, and the packet loss rate. Forexample, the moving image quality control unit 105 may determine thefirst correlation formula with the minimum square error method.

Then, the moving image quality control unit 105 determines the packetloss rate, which makes an inclination of a graph showing the secondcorrelation formula (that is, a derivative value of the correlationformula) zero, on the basis of the estimated second correlation formula.That is, the moving image quality control unit 105 determines the packetloss rate, which makes the encode bit rate of the moving image maximum,on the basis of the correlation relation between the encode bit rate andthe packet loss rate of the moving image. Then, the moving image qualitycontrol unit 105 determines redundancy of FEC on the basis of the packetloss rate.

Here, the redundancy means a value equal to subtraction of number ofbits, which are included in the packet and are indispensable as actualdata, from number of bits used to transfer the packet.

As another method to determine the redundancy of FEC, the moving imagequality control unit 105 may determine the redundancy of FEC with thefollowing method. The moving image quality control unit 105 may store atable, which associates the packet loss rate and the redundancy of FEC,in advance. Then, the moving image quality control unit 105 determinesthe redundancy of FEC on the basis of the packet loss rate withreference to the table. Moreover, the moving image quality control unit105 may store a table, which associates the packet loss rate and a FECcontrol method (for example, information which indicates a cyclic periodof FEC), in advance.

The moving image quality control unit 105 specifies finally that theencode bit rate of the moving image, which is corresponding to thedetermined packet loss rate (packet loss rate which makes the encode bitrate of the moving image maximum), is the encode bit rate of the movingimage which the moving image transmission apparatus 101 transmits.

Fourthly, the moving image quality control unit 105 calculates thetransmission bit rate of the moving image on the basis of the determinedencode bit rate of the moving image and the redundancy of FEC.Specifically, the moving image quality control unit 105 may calculatethe transmission bit rate of the moving image by adding a bit rate ofdata, which is determined on the basis of an amount of data necessaryaccording to the redundancy of FEC, to the encode bit rate of the movingimage.

Here, when the moving image quality control unit 105 determines theredundancy of FEC and the encode bit rate of the moving image, themoving image quality control unit 105 may adopt, as the determinedvalue, not the value which is calculated directly by the secondcorrelation formula, but an evaluated value which is equal tomultiplication of the calculated value by a weighting coefficient. Theweighting coefficient may be determined based on a difference betweenthe current redundancy of FEC and the newly-determined redundancy ofFEC, or a difference between the current encode bit rate of the movingimage and the newly-determined encode bit rate of the moving image. Inthis case, the moving image quality control unit 105 mat determine thatevaluated values, which are equal to multiplication of thenewly-determined values by the predetermined weighting coefficients, arethe redundancy of FEC and the encode bit rate of the moving image whichare to be applied after the control respectively.

Meanwhile, degree of the correlation between the transmission bit rateand the packet loss rate may be influenced in some cases by degree thatan amount of data, which is transmitted by the moving image transmissionapparatus 101, occupies the network 113. For example, in the case thatthe network 113 has quite a wide bandwidth, a change of the packet lossrate may have almost no influence on the transmission bit rate of thedata, which the moving image transmission apparatus 101 transmits, insome cases. In the case, the correlation between the transmission bitrate and the packet loss rate is weak. That is, the clear correlationbetween the packet loss rate and the transmission bit rate does notexist. In the case, the moving image quality control unit 105 may makethe control reflect the determined value slightly, even if the packetloss rate is determined. For example, the moving image quality controlunit 105 may carries out an evaluation of determining that an average ofthe current packet loss rate and the determined packet loss rate is anew packet loss rate.

On the other hand, in the case that the network 113 has a narrowbandwidth, the transmission bit rate of the data, which the moving imagetransmission apparatus 101 transmits, may be influenced severely by thechange of the packet loss rate, and may be changed significantly in somecases. In the case, the correlation between the transmission bit rateand the packet loss rate is strong. That is, quite a clear correlationrelation between the packet loss rate and the transmission bit rate isfound. In the case, the moving image quality control unit 105 maydetermine the packet loss rate, and make the control reflect thedetermined value strongly. For example, the moving image quality controlunit 105 may carry out an evaluation of determining that the new packetloss rate is equal to a value which is changed from the current packetloss rate by up to two times large differential-value of the currentpacket loss rate from the determined packet loss rate. It is possiblethat the moving image transmission system 1 carries out the precisecontrol, which is adaptive to the situation, by adding the evaluationmentioned above.

Moreover, the moving image quality control unit 105 may adjust the valueof the transmission bit rate, which is stored by the state informationrecord storing apparatus 104, on the basis of the calculatedtransmission bit rate and the transmission bit rate of the data which istransferred currently. For example, the calculated transmission bit ratemay be a target value which is achieved over a predetermined period oftime from the present time. That is, the moving image quality controlunit 105 may adjust the transmission bit rate, which is stored by thestate information record storing apparatus 104, so as to become close tothe calculated transmission bit rate gradually from the currenttransmission bit rate over the predetermined period of time.

According to the above-mentioned method, after the target value isachieved for a certain period of time, the value of the transmission bitrate, which is stored by the state information record storing apparatus104, is actually coincident with the calculated transmission bit rate ata later time. On the other hand, in the case that the target value ischanged within a certain period of time, the transmission bit rate,which is stored by the state information record storing apparatus 104,is adjusted at every time when the target value is changed so as tobecome close to the new target value gradually.

It is possible that the moving image transmission system 1 restrains anabrupt fluctuation of the moving image quality by virtue of the control.

Moving Image Encoding Unit 106

The moving image encoding unit 106 generates the moving image data byencoding the moving image on the basis of the encode bit rate which themoving image quality control unit 105 determines.

FEC Encoding Unit 107

The FEC encoding unit 107 generates the FEC data by use of theredundancy of FEC, which the moving image quality control unit 105determines, on the basis of the moving image data which the moving imageencoding unit 106 generates. The FEC encoding unit 107 is also called aredundant code encoding unit.

Here, the network state information which the state informationreceiving unit 103 receives may include the information which indicatesthe packet loss rate caused after the FEC decoding. In the case, thatthere is data which can not be decoded even by use of FEC indicates thatthere is a possibility that the continuous packet losses are caused inthe network which the moving image transmission system 1 uses. In thecase, when carrying out the FEC encoding, the FEC encoding unit 107 maycontrive a means such as selecting an algorism, which has robustnessagainst more continuous packet losses.

Data Transmitting Unit 108

The data transmitting unit 108 converts the moving image data, which themoving image encoding unit 106 generates, and the FEC data, which theFEC encoding unit 107 generates, into a data format which permits thetransmission via the network 113, and transmits the converted data. Thedata transmitting unit 108 transmits each the packet which includes theinformation indicating the packet. Furthermore, the data transmittingunit 108 makes the state information record storing apparatus 104 storethe transmission bit rate information. Here, the transmission bit rateinformation is generated by associating the transmission bit rate whichis used when transmitting each the packet, and the information whichindicates the time when the data is transmitted at the transmission bitrate.

The data transmitting unit 108 may assign the cyclic counter value toeach the packet which is transmitted.

Moving Image Receiving Apparatus 102

The moving image receiving apparatus 102 includes a data receiving unit109, a FEC decoding unit 110, a moving image decoding unit 111 and astate information transmitting unit 112.

Data Receiving Unit 109

The data receiving unit 109 receives the moving image and the FEC data,and calculates the packet loss rate caused before the FEC decoding.Moreover, the data receiving unit 109 transfers both of the informationwhich indicates the packet including the received data, and theinformation, which indicates the packet loss rate caused before the FECdecoding in the moving image receiving unit 102, to the stateinformation transmitting unit 112 which will be described later. Theinformation which indicates the packet can be substituted by informationwhich indicates a time interval from a point of time when the datareceiving unit 109 notifies the state information transmitting unit 112of the information which indicates the previous packet. Moreover, theinformation which indicates the packet loss rate caused before the FECdecoding in the moving image receiving apparatus 102 can be substitutedby information which indicates an amount of the moving image data andthe FEC data which are transmitted at a predetermined time.

Here, the information which indicates the time interval may be, forexample, information which indicates a length of a predetermined periodof time. Moreover, the information which indicates the time interval maybe, for example, a difference between a current time and a time when thestate information transmitting unit 112 transmits outward both of theinformation which indicates the previous packet, and the informationwhich indicates the packet loss rate caused before the FEC decoding inthe moving image receiving apparatus 102.

FEC Decoding Unit 110

The FEC decoding unit 110 restores the lost data on the basis of themoving image data and the FEC data which the data receiving unit 109receives, and re-generates the moving image data which is not caused thelost data yet.

Moreover, the FEC decoding unit 110 may notify the state informationtransmitting unit 112, which will be described later, of the informationwhich indicates the packet loss rate caused after the FEC decoding.

Moving Image Decoding Unit 111

The moving image decoding unit 111 plays back a moving picture and avoice by decoding the moving image data which the FEC decoding unit 110restores. In order to decode the moving picture and the voiceseparately, the moving image decoding unit 111 may have differentdecoding units which decode the moving picture and the voicerespectively.

State Information Transmitting Unit 112

From the data receiving unit 109, the state information transmittingunit 112 receives the information which indicates the packet, and theinformation which indicates the packet loss rate caused before the FECdecoding in the moving image receiving apparatus 102. Then, the stateinformation transmitting unit 112 transmits the information whichindicates the packet, and the information which indicates the packetloss rate caused before the FEC decoding outward through the network 114as the network state information.

The network state information may include the information whichindicates the packet loss rate caused after the FEC decoding.

FIG. 2 is a flowchart showing an outline of operations of the movingimage transmission apparatus 101 according to the first exemplaryembodiment of the present invention.

The moving image transmission apparatus 101 starts the operation (StepS201). The state information receiving unit 103 acquires the networkstate information on the network 113 via the network 114 (Step S202).The state information receiving unit 103 reads the transmission bit rateinformation of the moving image, which is generated at the timecorresponding to the information indicating the packet, from the stateinformation record storing apparatus 104 on the basis of theinformation, which indicates the packet, out of the received networkstate information. Then, the state information receiving unit 103 makesthe state information record storing apparatus 104 store the networkstate record information which includes both of the received networkstate information and the transmission bit rate included in the readtransmission bit rate information (Step S203).

The moving image quality control unit 105 reads the network state recordinformation from the state information record storing apparatus 104(Step S204). The moving image quality control unit 105 calculates thecorrelation between the transmission bit rate and the packet loss rate,which are recorded within a fixed period of time, on the basis of theread network state record information, and expresses the calculationresult as the first correlation formula (Step S205). The moving imagequality control unit 105 calculates the correlation between the encodebit rate and the packet loss rate of the moving image on the basis ofthe transmission bit rate which is determined by the first correlationformula, and expresses the calculation result as the second correlationformula (Step S206).

The moving image quality control unit 105 calculates the packet lossrate which makes an inclination (derivative value) of the determinedsecond correlating formula zero, and determines the redundancy of FEC onthe basis of the calculated packet loss rate. Moreover, the moving imagequality control unit 105 determines the encode bit rate of the movingimage, which is transmitted, on the basis of the encode bit ratecorresponding to the determined packet loss rate (Step S207).

The moving image encoding unit 106 encodes the moving image at theencode bit rate which is determined in Step S207 to generate the movingimage data (Step S208).

The FEC encoding unit 107 generates the FEC data by use of theredundancy of FEC, which is determined in Step S207, on the basis of themoving image data which the moving image encoding unit 106 generates(Step S209).

The data transmitting unit 108 converts the moving image data, which themoving image encoding unit 106 generates, and the FEC data, which theFEC encoding unit 107 generates, into a data format which permits thetransmission via the network 113, and transmits the converted data.Furthermore, the data transmitting unit 108 associates the transmissionbit rate which is used when transmitting the data, and the informationwhich indicates the time when the data is transmitted at thetransmission bit rate, and makes the state information record storingapparatus 104 store the association. (Step S210).

If another moving image to be transmitted still continues, the movingimage transmission apparatus 101 repeats the same steps which start fromStep S202 (Step S211). If any other moving image to be transmitted doesnot continue, the moving image transmission apparatus 101 ends theoperation (Step S212).

FIG. 3 is a flowchart showing an outline of operations of the movingimage receiving apparatus 102 according to the first exemplaryembodiment of the present invention.

The moving image receiving apparatus 102 starts the operation (StepS301). The data receiving unit 109 receives the moving image data andthe FEC data via the network 113, and transfers both the data to the FECdecoding unit 110. Moreover, the data receiving unit 109 notifies thestate information transmitting unit 112 of the information whichindicates the packet including the received data, and of the informationwhich indicates the packet loss rate caused before the FEC decoding inthe moving image receiving apparatus 102 (Step S302).

The FEC decoding unit 110 restores the lost data on the basis of themoving image data and the FEC data which are transferred by the datareceiving unit 109, and re-generates the moving image data which is notcaused the lost data yet (Step S303).

The moving image decoding unit 111 decodes the moving image data whichthe FEC decoding unit 110 generates, and plays back the moving pictureand the voice (Step S304).

The state information transmitting unit 112 transmits the network stateinformation outward via the network 114 (Step S305). The network stateinformation includes the information which indicates the packet receivedfrom the data receiving unit 109, and the information which indicatesthe packet loss rate caused before the FEC decoding in the moving imagereceiving apparatus 102.

In the case that the data receiving unit 109 continues to receive themoving image, the moving image receiving apparatus 102 repeats the stepswhich start from Step S302 (Step S306). In the case that the datareceiving unit 109 does not continue to receive the moving image, themoving image receiving apparatus 102 ends the operation (Step S307).

According to the first exemplary embodiment, the moving imagetransmission system 1 determines the packet loss rate, which makes theencode bit rate of the moving images maximum, on the basis of thecorrelation relation between the transmission bit rate and the packetloss rate of the data. Then, the moving image transmission system 1determines the redundancy of the redundant code on the basis of thedetermined packet loss rate. Therefore, according to the moving imagetransmission system 1, it is possible to obtain a remarkable effect thatthe moving image transmission system 1 is possible to determine theadequate number of the packets, which carry the redundant data, on thebasis of the correlation relation between the packet loss rate whichindicates the states of the network 113 and the network 114, and thetransmission bit rate of the data.

Description of Modified Example of Exemplary Embodiment

A modified example of the first exemplary embodiment will be describedin the following.

With regard to the first exemplary embodiment, when the firstcorrelation formula is estimated, the moving image quality control unit105 may estimate a situation that the packet loss, which is observed inthe network 113 and the network 114, is caused. Furthermore, the movingimage quality control unit 105 may operate according to the estimatedsituation that the packet loss is caused (first modified example of thefirst exemplary embodiment). On the basis of the correlation relationbetween the transmission bit rate and the packet loss rate, the movingimage quality control unit 105 classifies the situation that the packetloss is caused. Specifically, on the basis of conditions which thetransmission bit rate and the packet loss rate satisfy, the moving imagequality control unit 105 estimates the situation that the packet loss iscaused. Then, the moving image quality control unit 105 changes thetransmission bit rate, the redundancy of FEC, and the method ofdetermining the transmission bit rate of the moving image, which are tobe applied after the control, on the basis of the situation that thepacket loss is caused.

FIG. 4 and FIG. 5 are flowcharts each of which shows an operation of themoving image quality control unit 105 according to the first modifiedexample of the first exemplary embodiment. The moving image qualitycontrol unit 105 starts the operation (Step S801). The moving imagequality control unit 105 determines whether variance of the transmissionbit rate and variance of the packet loss rate are not larger thanpredetermined threshold values respectively (Step S802). In the casethat the variance of the transmission bit rate and the variance of thepacket loss rate are not larger than the predetermined threshold valuesrespectively (“Yes” in Step S802), the moving image quality control unit105 determines that the network 113 and the network 114 are in stablestates respectively. In this case, the moving image quality control unit105 determines that there is no problem in maintaining the currentstate. Next, the moving image quality control unit 105 determineswhether the maximum packet loss rate is not larger than a predeterminedthreshold value (Step S803). In the case that the maximum packet lossrate is not larger than the predetermined threshold value (“Yes” in StepS803), the moving image quality control unit 105 determines that thereis no problem in the current states of the network 113 and the network114. Then, the moving image quality control unit 105 determines that thecurrent transmission bit rate, the current redundancy of FEC, and thecurrent transmission bit rate of the moving image are the transmissionbit rate, the redundancy of FEC, and the encode bit rate of the movingimage which are to be applied after the control respectively (StepS804). Specifically, the moving image quality control unit 105determines that the current redundancy of FEC is the redundancy of FECwhich is to be applied after the control (Step S804-1). The moving imagequality control unit 105 determines that the current transmission bitrate is the transmission bit rate which is to be applied after thecontrol (Step S804-2). The moving image quality control unit 105determines that the current transmission bit rate of the moving image isthe encode bit rate of the moving image which is to be applied after thecontrol (Step S804-3). Here, this state is corresponding to a state thatthe variance of the transmission bit rate and the variance of the packetloss rate are small respectively (not larger than the threshold value).Then, the moving image quality control unit 105 ends the operation (StepS811).

In the case that the maximum packet loss rate is larger than thepredetermined threshold value (“No” in Step S803), the moving imagequality control unit 105 determines that the network 113 and the network114 are currently in unstable states respectively. Then, the movingimage quality control unit 105 determines that the redundancy of FEC,which can restore the moving image data even if the packet loss rate isa maximum, is the redundancy of FEC which is to be applied after thecontrol (Step S805-1). Moreover, the moving image quality control unit105 determines that the current transmission bit rate is thetransmission bit rate which is to be applied after the control (StepS805-2). On the basis of the redundancy of FEC and the transmission bitrate which are to be applied after the control, the moving image qualitycontrol unit 105 determines the encode bit rate of the moving imagewhich is to be applied after the control (Step S805-3). A combination ofStep S805-1 to Step S805-3 is defined as Step S805. Here, the state iscorresponding to a state that both of the variance of the transmissionbit rate and the variance of the packet loss are large (not smaller thanthreshold value), as shown in FIG. 7. Then, the moving image qualitycontrol unit 105 ends the operation (Step S811).

In the case that any one of the variance of the transmission bit rateand the variance of the packet loss rate is larger than thepredetermined threshold value (“No” in Step S802), the moving imagequality control unit 105 determines whether a positive correlationrelation between the transmission bit rate and the packet loss rateexists (Step S806). In the case that the positive correlation relationbetween the transmission bit rate and the packet loss rate exists (“Yes”in Step S806), the moving image quality control unit 105 determines thatthe packet loss is caused due to own communication. In this case, themoving image transmission apparatus 101 can transmit the moving imagewith the best quality through the moving image quality control unit 105carrying out the quality control which is described in the firstexemplary embodiment of the present invention. The moving image qualitycontrol unit 105 carries out the operations of Step S206 and Step S207according to the first exemplary embodiment (Step S807-1 and StepS807-2). Then, the moving image quality control unit 105 determines thetransmission bit rate, the redundancy of FEC, and the encode bit rate ofthe moving image, which are to be applied after the control, by carryingout the above mentioned steps (Step S807). Here, the state iscorresponding to a state that the transmission bit rate and the packetloss rate are in a state shown in FIG. 8. Then, the moving image qualitycontrol unit 105 ends the operation (Step S811).

In the case that the positive correlation relation between thetransmission bit rate and the packet loss rate does not exist (“No” inStep S806), the moving image quality control unit 105 determines whethera correlation relation between the time and the packet loss rate exists(Step S808). In the case that the correlation relation between the timeand the packet loss rate exists (“Yes” in Step S808), the moving imagequality control unit 105 determines that other communication influencesthe packet loss. Then, the moving image quality control unit 105determines a correlation formula between the time and the packet lossrate, and estimates the packet loss rate which is caused after elapse ofa predetermined time (Step S809-1). On the basis of the estimated packetloss rate and the first correlation formula, the moving image qualitycontrol unit 105 determines the transmission bit rate which is to beapplied after the control (Step S809-2). Then, the moving image qualitycontrol unit 105 determines that the redundancy of FEC, which canrestore the moving image data even in the case of the estimated packetloss rate, is the redundancy of FEC which is to be applied after thecontrol (Step S809-3). Furthermore, on the basis of the redundancy ofFEC and the transmission bit rate which are to be applied after thecontrol, the moving image quality control unit 105 determines the encodebit rate of the moving image which is to be applied after the control(Step S809-4). A combination of Step S809-1 to Step S809-4 is defined asStep S809. Here, the state is corresponding to a state that thetransmission bit rate and the packet loss rate are in a state shown inFIG. 9. Then, the moving image quality control unit 105 ends theoperation (Step S811).

In the case that the correlation relation between the time and thepacket loss rate does not exist (“No” in Step S808), the moving imagequality control unit 105 determines that it is impossible to estimatethe situation that the packet loss is caused. In this case, the movingimage quality control unit 105 does not carry out the control. Themoving image quality control unit 105 determines that the currenttransmission bit rate, the current redundancy of FEC, and the currenttransmission bit rate of the moving image are the transmission bit rate,the redundancy of FEC, and the encode bit rate of the moving image whichare to be applied after the control respectively (Step S810).Specifically, the moving image quality control unit 105 determines thatthe current redundancy of FEC is the redundancy of FEC which is to beapplied after the control (Step S810-1). The moving image qualitycontrol unit 105 determines that the current transmission bit rate isthe transmission bit rate which is to be applied after the control (StepS810-2). The moving image quality control unit 105 determines that thecurrent transmission bit rate of the moving image is the encode bit rateof the moving image which is to be applied after the control (StepS810-3). Here, the state is corresponding to a state that thetransmission bit rate and the packet loss rate are in a state shown inFIG. 10. Then, the moving image quality control unit 105 ends theoperation (Step S811).

Note, that according to the flowchart mentioned above, the moving imagequality control unit 105 determines the transmission bit rate, theredundancy of FEC, and the transmission bit rate of the moving image,which are to be applied after the control, by use of one method on thebasis of the transmission bit rate and the packet loss. The moving imagequality control unit 105 may determine the transmission bit rate, theredundancy of FEC, and the transmission bit rate of the moving image,which are to be applied after the control, by use of a plurality ofmethods on the basis of the transmission bit rate and the packet loss.Then, the moving image quality control unit 105 may multiply values,which are determined by use of a plurality of the methods, by weightingcoefficients respectively, and determines that an average of themultiplied values is a desired value.

For example, the moving image quality control unit 105 may determine thetransmission bit rate in a way described in the following. Thetransmission bit rate which is determined in Step S805 mentioned aboveis denoted as B5, and the transmission bit rate which is determined inStep S807 mentioned above is denoted as B7, and the transmission bitrate which is determined in Step S809 mentioned above is denoted as B9.In this case, the moving image quality control unit 105 may determine atransmission bit rate B on the basis of [formula 1] described in thefollowing. Here, each of α1, α2 and α3 is a predetermined real number.B=B ₅×α₁ +B ₇×α₂ +B ₉×α₃  [Formula 1]where α1, α2 and α3 may be values determined in advance or may be valuesdetermined by learning. Moreover, α1, α2 and α3 may be determined on thebasis of the situation that the packet loss is caused. The situationthat the packet loss is caused is estimated by the moving image qualitycontrol unit 105.

According to the first modified example of the first exemplaryembodiment, the moving image transmission system 1 determines thecorrelation relation between the transmission bit rate and the packetloss rate of the data. Then, on the basis of the determined correlationrelation between the transmission bit rate and the packet loss rate, themoving image transmission system 1 estimates the situation that thepacket loss is caused. Therefore, according to the moving imagetransmission system 1, it is possible to obtain a remarkable effect thatthe moving image transmission system 1 determines the adequate number ofthe packets, which carry the redundant data, on the basis of thecorrelation relation between the packet loss rate which indicates thestates of the network 113 and the network 114, and the transmission bitrate of the data.

Second Exemplary Embodiment

FIG. 11 is a block diagram showing an exemplary configuration of amoving image transmission system 4 according to a second exemplaryembodiment of the present invention.

The moving image transmission system 4 includes a moving imagetransmission server 401 and a moving image receiving terminal 402. Themoving image transmission server 401 and the moving image receivingterminal 402 are connected each other via the internet 413 and theinternet 414. Only a different point of the second exemplary embodimentfrom the first exemplary embodiment will be focused and describedhereinafter. A component according to the second exemplary embodiment,which has the same configuration as the component according to the firstexemplary embodiment has, is assigned the same numeral number as thecomponent according to the first exemplary embodiment is assigned, anddescription on the component is omitted.

The moving image transmission server 401 includes a state informationreceiving module 403, a state information record storing memory 404, amoving image quality control module 405, a MPEG-2 encoder 406, a FECencoder 407 and an UDP transmitting module 408.

The moving image receiving terminal 402 includes an UDP receiving module409, a FEC decoder 410, a MPEG-2 decoder 411 and a state informationtransmitting module 412.

It is assumed that the FEC encoder 407 and the FEC decoder 410 use theLDPC-FEC (Low Density Parity Check Forward Error Correction) method asthe FEC method.

FIG. 12 is a sequence diagram showing an outline of operations of themoving image transmission server 401 according to the second exemplaryembodiment of the present invention.

The moving image transmission server 401 starts the operation (StepS501). The state information receiving module 403 acquires network stateinformation on the internet 413 via the internet 414 (Step S502). It isassumed that the network state information includes a time stamp valuewhich indicates a time when each packet is transmitted by the movingimage transmission server 401, a cyclic counter value, information whichindicates a packet loss rate caused before FEC decoding, and informationwhich indicates the packet loss rate after the FEC decoding. On thebasis of the time stamp value which is included in the received networkstate information, the state information receiving module 403 readstransmission bit rate information of a moving image, which is generatedat a time corresponding to the time stamp value, from the stateinformation record storing memory 404. Then, the state informationreceiving module 403 makes the state information record storing memory404 store network state record information (Step S503). The networkstate record information includes the network state information whichthe state information receiving module 403 receives, and a transmissionbit rate which is included in the read transmission bit rateinformation. The transmission bit rate information is stored by thestate information record storing memory 404 in Step S507 which is thepast step.

The moving image quality control module 405 reads the network staterecord information from the state information record storing memory 404(Step S504). The moving image quality control module 405 calculates acorrelation between the transmission bit rate and the packet loss rate,which are recorded within a fixed period of time, on the basis of thenetwork state record information which is read, and expresses thecalculation result as the first correlation formula (Step S505). It isassumed that the state information record storing memory 404 stores thenetwork state record information which is acquired every almost onesecond. Moreover, it is assumed that the moving image quality controlmodule 405 reads five past records. Number of the records and the methodfor acquiring the record are merely for an exemplified configuration inthis embodiment. The acquisition interval may be more lengthened.Moreover, the moving image quality control module 405 may multiplyinformation of the record which is acquired at a short interval andinformation of the record which is acquired at a long interval byweighting coefficients respectively, and uses a combination of both theweighted information.

Here, the packet loss rate is denoted as x and the transmission bit rateis denoted as y. It is possible to estimate that the first correlationformula is expressed, for example, by an approximate formula shown inFIG. 13 on the basis of the five record data by use of the minimumsquare error method. A form of this graph indicates the following. Thatis, FIG. 13 shows that, in the case that the transmission bit ratebecomes large to some extent, an available bandwidth decreases and thepacket loss rate increases.

For example, in the case that the approximate formula shown in FIG. 13is approximated by a first order formula of x, it is estimated that thefirst correlation formula is expressed as an approximate formula y=ax+b(a and b are fixed numbers calculated with the minimum square errormethod). Moreover, in the case that the approximate formula shown inFIG. 13 is expressed by use of logarithm of x, it is estimated that thefirst correlation formula is expressed as an approximate formulay=a*(log x)+b (a and b are fixed numbers calculated with the minimumsquare error method). Moreover, in the case that the approximate formulashown in FIG. 13 is expressed by use of a reciprocal of x, it isestimated that the first correlation formula is expressed as anapproximate formula y=a(1−b/x)+b (a and b are fixed numbers calculatedwith the minimum square error method).

The moving image quality control module 405 determines an encode bitrate of the moving image, which the moving image quality control module405 transmits, on the basis of the transmission bit rate which isdetermined by the calculated first correlation formula (Step S506). Themoving image quality control module 405 determines the encode bit rateof the moving image on the basis of the determined transmission bitrate, a header size of the moving image data at the transmission bitrate, and a data size of the FEC data. Specifically, the moving imagequality control module 405 determines the encode bit rate of the movingimage by subtracting a bit rate which is corresponding to the data sizeof the header, and a bit rate which is corresponding to the data size ofthe FEC data from the transmission bit rate. It is assumed that thetransport layer of MPEG-2 (Moving Picture Experts Group 2), which isused as the format of the moving image, is MPEG-2 TS (Transport Stream)in order to mix and transmit a moving picture and a voice. In this case,an amount of the header data is coincident with a total amount of the IP(Internet Protocol) header data, the UDP (User Datagram Protocol) headerdata and the MPEG-2 TS header data. In this case, an amount of the FECdata is coincident with a total amount of payload data of the cyclic FECpacket which has sufficient redundancy against the targeted packet lossrate, and the FEC header data.

The moving image quality control module 405 estimate a secondcorrelation formula, which indicates a correlation relation between theencode bit rate and the packet loss rate of the moving image, on thebasis of the determined encode bit rate and the determined firstcorrelation formula of the moving image. For example, it can beestimated that the second correlation formula is expressed as anapproximate formula shown in FIG. 14 by use of the minimum square errormethod. A form of this graph means the following. That is, FIG. 14 showsthat the transmission bit rate becomes the largest if the packet lossrate is coincident with a predetermined packet loss rate (packet lossrate which makes an inclination of the second correlation formula zero).The form of the graph shown in FIG. 14 means that the encode bit ratebecome decreasing since the redundant code occupies a large part of themoving image data if the packet loss rate become not smaller than apredetermined value.

For example, in the case that the second correlation formula isapproximated by a second order formula y=ax*x+bx+c (x*x means square ofx), the packet loss rate, which is corresponding to a peak point of thesecond order formula, makes an inclination of the correlation formulazero. The packet loss rate, which makes the inclination of thecorrelation formula zero, makes the encode bit rate maximum. The movingimage quality control module 405 determines the encode bit rate at whichthe moving image data transmitted on the basis of the maximized encodebit rate by the moving image transmission server 401 is encoded. By useof the second correlation formula, the moving image quality controlmodule 405 determines the packet loss rate which is estimated in thecase that the determined encode bit rate of the moving image data isused. Then, the moving image quality control module 405 determines theredundancy of FEC which can restore the data even in the case that theestimated packet loss rate is caused. The moving image quality controlmodule 405 makes the state information record storing memory 404 recordthe transmission bit rate, which is determined on the basis of theredundancy of FEC, together with information indicating a time when datais transmitted at the transmission bit rate (Step S507).

The MPEG-2 encoder 406 encodes the moving image at the encode bit ratewhich is determined in Step S507 and generates a MPEG-2 TS packet (StepS508).

The FEC encoder 407 generates a FEC packet by use of the redundancy ofFEC, which is determined in Step S507, on the basis of the MPEG-2 TSpacket which is generated by the MPEG-2 encoder 406 (Step S509).Moreover, the network state information which the state informationreceiving module 404 receives may include information indicating anamount of data, which can not be restored even by use of FEC, in somecases. In the case, the moving image quality control module 405determines that the continuously-lost packets whose length is longerthan an estimated length are caused in the internet 413. Then, in orderto select an algorithm which has robustness against more continuouspacket losses, the moving image quality control module 405 makes acyclic period of FEC lengthened and increases number of FEC streams. Thecyclic period of FEC means number of the packets (frame length) ofsource data to which the redundant code is added. The number of the FECstreams means number of the cyclic periods in the case that a pluralityof redundant codes are used at a plurality of the cyclic periods of FEC.

For example, in the case that the moving image quality control module405 carries out the FEC encoding currently with using two cyclic periodsof FEC, the moving image quality control module 405 may carry out theFEC encoding with using three cyclic periods of FEC. Moreover, themoving image quality control module 405 may change two cyclic periods toa set of the cyclic periods which are lengthened.

The UDP transmitting module 408 transmits the MPEG-2 TS packet which isgenerated in Step S508, and the FEC packet which is generated in StepS509 as an UDP packet via the internet 413 (Step S510).

In the case that the moving image transmission server 401 does notreceive a request such as a transmission halting command from an user ofthe moving image receiving terminal 402, the moving image transmissionserver 401 repeats the same steps which start from Step S502 (StepS511). Moreover, in the case that the moving image transmission server401 does not transmit up to an end of the moving image yet, the movingimage transmission server 401 repeats the same steps which start fromStep S502 (Step S511). In an else case, the moving image transmissionserver 401 ends the operation (Step S512). For example, in the case thatthe moving image transmission server 401 receives a request to jump to aspecific playback position, the moving image transmission server 401 maynot stop the operation (does not carry out the operation of Step S512),and changes a transmission position in the moving image data and startstransmitting the moving image data from the new transmission point.Then, the moving image transmission server 401 may carry out theoperations, which follow Step S511, at a later time.

FIG. 15 is a sequence diagram showing an outline of operations of themoving image receiving terminal 402 according to the second exemplaryembodiment of the present invention.

The moving image receiving terminal 402 starts the operation (StepS601). Then, the UDP receiving module 409 receives the MPEG-2 TS packetand the FEC packet via the internet 413. At this time, the UDP receivingmodule 409 notifies the state information transmitting module 412 of thetime stamp value of the received packet, the cyclic counter value of thereceived packet and number of packets which are lost when the movingimage receiving terminal 402 receives the packets (Step S602).

If there is the lost packet, the FEC decoder 410 restores the lostpacket, and re-generates the MPEG-2 TS packet, which is not caused thelost packet yet, on the basis of the MPEG-2 TS packet and the FEC packetwhich the UDP receiving module 409 receives in Step S602 (Step S603).Moreover, the FEC decoder 410 notifies the state informationtransmitting module 412 of the number of packets which can not berestored even by use of FEC.

The MPEG-2 decoder 411 decodes the MPEG-2 TS packet which is restored inStep S603, and plays back the moving picture and the voice (Step S604).

The state information transmitting module 412 transmits the networkstatus information outward via the internet 414 every almost one secondby use of a timer (Step S605). The network state information includesthe time stamp value of the last packet of the received data out of thepackets which are received within a period of time measured by thetimer, the cyclic counter value and number of the packets which are lostwithin the period of time.

If another moving image to be received still continues, the moving imagereceiving terminal 402 repeats the same steps which start from Step S602(Step S606). If another moving image to be received continues no longer,the moving image receiving terminal 402 ends the operation (Step S607).

Note, that the network, which is denoted as the internet according tothe first exemplary embodiment, may be a cellular phone network (3G,3.5G, 3.9G, 4G or the like), or the intranet. The communication packetmay be a RTP (Real-time Transport Protocol) packet or a packet, to whichanother transport protocol is applied, instead of the MPEG-2 TS packet.Formats of the moving picture and the voice have no restriction inparticular. Any kind of the format such as MPEG-4, H.263, H.264 or thelike may be applicable in addition to MPEG-2, and furthermore theformats of the moving picture and the voice based on thenon-standardized original coding method may be also applicable. Theformat of the voice has no restriction in particular. Any kind of theformat of the voice such as AAC (Advanced Audio Coding), G.711, G.722 orthe like may be applicable. Moreover, any kind of the terminal such as acellular phone, STB (Set Top Box), TV (Television), PC (PersonalComputer) or the like may be applicable.

According to the second exemplary embodiment, the moving imagetransmission system 4 determines the packet loss rate, which makes theencode bit rate of the moving image maximum, on the basis of thecorrelation relation between the transmission bit rate and the packetloss rate of the data. Then, the moving image transmission system 4determines the redundancy of the redundant code on the basis of thedetermined packet loss rate. Therefore, according to the moving imagetransmission system 4, it is possible to obtain a remarkable effect thatthe moving image transmission system 4 is possible to determine adequatenumber of the packets, which carry the redundant data, on the basis ofthe correlation relation between the packet loss rate which indicatesthe states of the internet 413 and the internet 114, and thetransmission bit rate of the data.

Description of Modified Example of Exemplary Embodiment

A modified example of the second exemplary embodiment will be describedin the following.

FIG. 16 is a block diagram showing an exemplary configuration of amoving image transmission system 7 according to the modified example ofthe second exemplary embodiment of the present invention.

The basic configuration of the moving image transmission system 7 shownin FIG. 16 is the same as one of the moving image transmission system 4shown in FIG. 11. A video conference terminal 701 and a video conferenceterminal 702, which are included in the moving image transmission system7 shown in FIG. 16, have also the function of the moving imagetransmission server 401 and the function of the moving image receivingterminal 402 shown in FIG. 16 respectively.

An operation of the moving image transmission system 7 is similar to theoperation of the moving image transmission system 4 according to thesecond exemplary embodiment except that each apparatus shown in FIG. 16has the transmitting and receiving function.

Third Exemplary Embodiment

FIG. 17 is a block diagram showing an exemplary configuration of aquality control apparatus 901 according to a third exemplary embodimentof the present invention.

The quality control apparatus 901 includes a state information receivingunit 903, a state information record storing unit 904 and a moving imagequality control unit 905.

The quality control apparatus 901 is connected with the outside via anetwork 914.

From the network 914, the state information receiving unit 903 receivesnetwork state information including both of information which indicatesa packet of a moving image data, and information which indicates apacket loss rate of the moving image data. Here, the information whichindicates the packet of the moving image data, and information whichindicates the packet loss rate of the moving image data are transmittedby a transmission unit which transmits the moving image data at apredetermined transmission bit rate.

The state information record storing unit 904 stores bit rateinformation, which includes the transmission bit rate of the movingimage data transmitted by the transmission unit, in association with thenetwork state information which the state information receiving unit 903receives.

Firstly, the moving image quality control unit 905 reads the informationwhich indicates the packet loss rate of the packet transmitted by thetransmission unit, and the bit rate information of the packet from thestate information record storing unit 904. Secondly, the moving imagequality control unit 905 estimates a correlation between an encode bitrate and the packet loss rate of the moving image data on the basis ofthese read information. Thirdly, the moving image quality control unit905 determines the packet loss rate, which makes the encode bit rate ofthe moving image data maximum, on the basis of the estimatedcorrelation. Fourthly, the moving image quality control unit 905determines redundancy, which is applied to the moving image data, on thebasis of the determined packet loss rate.

The quality control apparatus 901 shown in FIG. 17 carries out the sameoperation as the moving image transmission apparatus 101 according tothe first exemplary embodiment of the present invention carries out. Thequality control apparatus 901 determines the packet loss rate, whichmakes the encode bit rate of the moving image data maximum, on the basisof the correlation between the transmission bit rate and the packet lossrate of the data. Then, the quality control apparatus 901 determinesredundancy of a redundant code on the basis of the determined packetloss rate. Therefore, the quality control apparatus 901 according to thethird exemplary embodiment of the present invention, can obtain the sameeffect as the moving image transmission apparatus 101 according to thefirst exemplary embodiment of the present invention obtains.

While the invention of the present application has been described withreference to the exemplary embodiment and the example, the invention ofthe present application is not limited to the above-mentioned exemplaryembodiment and the example. It is possible to make various changes,which a person skilled in the art can understand, in the form anddetails of the invention of the present application without departingfrom the sprit and scope of the invention of the present application.

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2009-266018, filed on Nov. 24, 2009, thedisclosure of which is incorporated in its entirety by reference.

INDUSTRIAL APPLICABILITY

The moving image transmission apparatus according to the presentinvention can be applied to an application which uses the moving imageand which is provided through an unstable network. For example, a videoconference, a video phone, broadcast, VoD (Video on Demand), nPVR(network Personal Video Recorder) or the like is exemplified as theapplication which uses the moving image.

DESCRIPTION OF THE CODES

-   -   1 moving image transmission system    -   4 moving image transmission system    -   7 moving image transmission system    -   101 moving image transmission apparatus    -   102 moving image receiving apparatus    -   103 state information receiving unit    -   104 state information record storing apparatus    -   105 moving image quality control unit    -   106 moving image encoding unit    -   107 FEC encoding unit    -   108 data transmitting unit    -   109 data receiving unit    -   110 FEC decoding unit    -   111 moving image decoding unit    -   112 state information transmitting unit    -   113 network    -   114 network    -   401 moving image transmission server    -   402 moving image receiving terminal    -   403 State information receiving module    -   404 state information record storing memory    -   405 moving image quality control module    -   406 MPEG-2 encoder    -   407 FEC encoder    -   408 UDP transmitting module    -   409 UDP receiving module    -   410 FEC decoder    -   411 MPEG-2 decoder    -   412 state information transmitting module    -   413 internet    -   414 internet    -   701 video conference terminal    -   702 video conference terminal    -   901 quality control apparatus    -   903 state information receiving unit    -   904 state information record storing apparatus    -   905 moving image quality control unit    -   914 network

The invention claimed is:
 1. A quality control apparatus, comprising: astate information receiving unit to receive network state informationwhich includes both of a packet loss rate of moving image data caused ata certain time and time information indicating the certain time; a stateinformation record storing apparatus to store bit rate information,which includes both of a transmission bit rate of the moving image datacaused at a certain time and time information indicating the certaintime, in association with the network state information which includesthe time information; a moving image quality control unit to read bothof information indicating the packet loss rate which is associated withthe time information included in each packet of the moving image dataand the transmission bit rate from the state information record storingapparatus, and to estimate a correlation between an encode bit rate andthe packet loss rate of the moving image data based on each of the readinformation, and to determine the packet loss rate, which makes theencode bit rate of the moving image data maximum based on thecorrelation, and to determine redundancy, which is applied to the movingimage data based on the determined packet loss rate, wherein the movingimage quality control unit estimates a first correlation formula, whichrepresents a correlation between the transmission bit rate and thepacket loss rate, based on the information indicating the packet lossrate and the transmission bit rate which are read from the stateinformation record storing device, and estimates a second correlationformula, which represents the correlation between the encode bit rateand the packet loss rate of the moving image data based on a differencebetween the transmission bit rate which is corresponding to theinformation indicating each the packet loss rate and which is determinedby the first correlation formula and a transmission bit rate of aredundant code, which is used at a time indicated by the timeinformation included in the bit rate information including thetransmission bit rate corresponding to the information indicating eachthe packet loss rate and which is determined based on the packet of themoving image data, and determines the packet loss rate, which makes theencode bit rate of the moving image data maximum based on the secondcorrelation formula, and determines the redundancy, which is applied tothe moving image data, based on the determined packet loss rate.
 2. Thequality control apparatus according to claim 1, wherein the moving imagequality control unit adjusts the determined redundancy based on networkbandwidth which the quality control apparatus uses.
 3. The qualitycontrol apparatus according to claim 1, wherein the moving image qualitycontrol unit adjusts the determined redundancy based on a differencebetween the transmission bit rate included in the bit rate informationwhich includes the time information included in each the packet of themoving image data and a transmission bit rate which is determined basedon the maximum encode bit rate and the determined redundancy of thedata.
 4. The quality control apparatus according to claim 1, whereinbased on a determination whether there is a correlation relation betweenthe packet loss rate and the time information which are included in thenetwork state information, the moving image quality control unitdetermines whether the redundancy applied to the moving image data isadjusted.
 5. A quality control apparatus, comprising: a stateinformation receiving unit to receive network state information whichincludes both of a packet loss rate of moving image data caused at acertain time and time information indicating the certain time; a stateinformation record storing apparatus to store bit rate information,which includes both of a transmission bit rate of the moving image datacaused at a certain time and time information indicating the certaintime, in association with the network state information which includesthe lime information, a moving image quality control unit to read bothof information indicating the packet loss rate which is associated withthe time information included in each packet of the moving image dataand the transmission bit rate from the state information record storingapparatus, and to estimate a correlation between an encode bit rate andthe packet loss rate of the moving image data based on each of the readinformation, and to determine the packet loss rate, which makes theencode bit rate of the moving image data maximum based on thecorrelation, and to determine redundancy, which is applied to the movingimage data based on the determined packet loss rate, wherein the movingimage quality control unit determines a transmission bit rate whichexists between the transmission bit rate included in the bit rateinformation which includes the time information included in each thepacket of the moving image data and the transmission bit rate which isdetermined based on the maximum encode bit rate and the determinedredundancy of the data, and determines new redundancy based on thedetermined transmission bit rate and the maximum encode bit rate.
 6. Thequality control apparatus according to claim 5, wherein after apredetermined period of time elapses, the moving image quality controlunit adjusts the new redundancy so as to be coincident with redundancywhich is determined based on the packet loss rate corresponding to themaximum encode bit rate.
 7. A quality control apparatus, comprising: astate information receiving unit to receive network state informationwhich includes both of a packet loss rate of moving image data caused ata certain time and time information indicating the certain time; a stateinformation record storing apparatus to store bit rate information,which includes both of a transmission bit rate of the moving image datacaused at a certain time and time information indicating the certaintime, in association with the network state information which includesthe time information; a moving image quality control unit to read bothinformation indicating the packet loss rate which is associated with thetime information included in each packet of the moving image data andthe transmission bit rate from the state information record storingapparatus, and to estimate a correlation between an encode hit rate andthe packet loss rate of the moving image data based on each of the readinformation, and to determine the packet loss rate, which makes theencode bit rate of the moving image data maximum based on thecorrelation, and to determine redundancy, which is applied to the movingimage data based on the determined packet loss rate based on adetermination whether at least one out of variance of the packet lossrate which is included in the network state information, variance of thetransmission bit rate which is included in the bit rate information, andthe maximum value of the packet loss rate is not larger than apredetermined threshold value, the moving image quality control unitdetermines whether the redundancy applied to the moving image data isadjusted.
 8. A quality control apparatus, comprising: a stateinformation receiving unit to receive network state information whichincludes both of a packet loss rate of moving image data caused at acertain time and time information indicating the certain time, a stateinformation record storing apparatus to store bit rate information,which includes both of a transmission bit rate of the moving image datacaused at a certain time and time information indicating the certaintime, in association with the network state information which includesthe time information; a moving image quality control unit to read bothof information indicating the packet loss rate which is associated withthe time information included in each packet of the moving image dataand the transmission bit rate from the state information record storingapparatus, and to estimate a correlation between an encode bit rate andthe packet loss rate of the moving image data based on each of the readinformation, and to determine the packet loss rate, which makes theencode bit rate of the moving image data maximum based on thecorrelation, and to determine redundancy, which is applied to the movingimage data based on the determined packet loss rate, a redundant codeencoding unit which adds a redundant code to the moving image data basedon the redundancy which the moving image quality control unit determinesor adjusts, wherein the network state information includes informationwhich indicates the packet loss rate caused after decoding and which iscorresponding to a ratio of an amount of data unable to be decoded evenif the redundant code decodes the moving image data to an amount of themoving image data, and the redundant code encoding unit changes analgorithm, which generates the redundant code based on the informationwhich indicates the packet loss rate caused after the decoding.
 9. Thequality control apparatus according to claim 8, comprising: a movingimage encoding unit to encode the moving image data based on the packetloss rate which the moving image quality control unit determines; and adata transmitting unit to transmit the moving image data to which theredundant code encoding unit adds the redundant code, wherein the movingimage quality control unit makes the state information record storingapparatus store the bit rate information including both of thetransmission bit rate which is determined based on the maximum encodebit rate of the moving image data and information which indicates arange of time when the moving image data is transmitted at thetransmission bit rate.
 10. A moving image transmission system,comprising: a moving image transmission apparatus; and a moving imagereceiving apparatus, wherein the moving image transmission apparatusincludes: a transmission unit to transmit moving image data at apredetermined transmission bit rate, a state information receiving unitto receive network state information which includes both of a packetloss rate of the moving image data caused at a certain time and timeinformation indicating the certain time, a state information recordstoring apparatus to store bit rate information, which includes both ofthe transmission bit rate of the moving image data used at a certaintime and time information indicating the certain time, in associationwith the network state information which includes the time information,and a moving image quality control unit to read both of the informationindicating the packet loss rate which is associated with the timeinformation included in each packet of the moving image data and thetransmission bit rate from the state information record storingapparatus, and to estimate a correlation between an encode bit rate andthe packet loss rate of the moving image data base on each of the readinformation, and to determine the packet loss rate, which makes theencode bit rate of the moving image data maximum based on thecorrelation, and to determine redundancy, which is applied to the movingimage data based on the determined packet loss rate, wherein the movingimage quality control unit estimates a first correlation formula, whichrepresents a correlation between the transmission bit rate and thepacket loss rate, based on the information indicating the packet lossrate and the transmission bit rate which are read from the stateinformation, record storing apparatus, and estimates a secondcorrelation formula, which represents the correlation between the encodebit rate and the packet loss rate of the moving image data based on adifference between the transmission bit rate which is corresponding tothe information indicating each the packet loss rate and which isdetermined by the first correlation formula and a transmission bit rateof a redundant code, which is used at a time indicated by the timeinformation included in the bit rate information including thetransmission bit rate corresponding to the information indicating eachthe packet loss rate and which is determined based on the packet of themoving image data, and determines the packet loss rate, which makes theencode bit rate of the moving image data maximum based on the secondcorrelation formula, and determines the redundancy, which is applied tothe moving image data, based on the determined packet loss rate; themoving image receiving apparatus includes: a data receiving unit toreceive the moving image data from the moving image transmissionapparatus; and a state information transmitting unit to transmit both ofthe packet loss rate of the moving image data, and time informationindicating a time, at which the moving image transmission apparatustransmits each the packet included in the moving image data, to themoving image transmission apparatus.
 11. A quality control method,comprising: receiving network state information which includes both of apacket loss rate of moving image data caused at a certain time and timeinformation indicating the certain time; making a storage medium storebit rate information, which includes both of a transmitting bit rate ofthe moving image data used at a certain time and time informationindicating the certain time, in association with the network stateinformation which includes the time information; reading both of theinformation indicating the packet loss rate, which is associated withthe time information included in each packet of the moving image dataand the transmission bit rate from the storage medium; estimating acorrelation between an encode bit rate and the packet loss rate of themoving image data based on each of the read information; determining thepacket loss rate, which makes the encode bit rate of the moving imagemaximum based on the correlation; determining redundancy, which isapplied to the moving image data based on the determined packet lossrate; estimating a first correlation formula, which represents acorrelation between the transmission bit rate and the packet loss rate,based on the information indicating the packet loss rate and thetransmission bit rate; estimating a second correlation formula, whichrepresents the correlation between the encode bit rate and the packetloss rate of the moving image data based on a difference between thetransmission bit rate which is corresponding to the informationindicating each the packet loss rate and which is determined by thefirst correlation formula and a transmission bit rate of a redundantcode, which is used at a time indicated by the time information includedin the bit rate information including the transmission bit ratecorresponding to the information indicating each the packet loss rateand which is determined based on the packet of the moving image data;determining the packet loss rate, which makes the encode bit rate of themoving image data maximum based on the second correlation formula; anddetermining the redundancy, which is applied to the moving image data,based on the determined packet loss rate.
 12. A non-transitory recordingmedium storing a program to makes a computer execute: a process toreceive network state information which includes both of a packet lossrate of moving image data caused at a certain time and time informationindicating the certain time; a process to make a storage medium storebit rate information, which includes both of a transmitting bit rate ofthe moving image data used at a certain time and time informationindicating the certain time, in association with the network stateinformation which includes the time information; a process to read bothof the information indicating the packet loss rate, which is associatedwith the time information included in each packet of the moving imageand the transmission bit rate; a process to estimate a correlationbetween an encode bit rate and the packet loss rate of the moving imagedata based on each of the read information; a process to determine thepacket loss rate, which makes the encode bit rate of the moving imagemaximum based on the correlation; a process to determine redundancy,which is applied to the moving image data based on the determined packetloss rate a process to estimate a first correlation formula, whichrepresents a correlation between the transmission bit rate and thepacket loss rate, based on the information indicating the packet lossrate and the transmission bit rate; a process to estimate a secondcorrelation formula, which represents the correlation between the encodebit rate and the packet loss rate of the moving image data based on adifference between the transmission bit rate which is corresponding tothe information indicating each the packet loss rate and which isdetermined by the first correlation formula and a transmission bit rateof a redundant code, which is used at a time indicated by the timeinformation included in the bit rate information including thetransmission bit rate corresponding to the information indicating eachthe packet loss rate and which is determined based on the packet of themoving image data; a process to determine the packet loss rate, whichmakes the encode bit rate of the moving image data maximum based on thesecond correlation formula; and a process to determine the redundancy,which applied to the moving image data, based on the determined packetloss rate.